{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'3.9.2'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import keras\n",
    "keras.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text generation with LSTM\n",
    "\n",
    "This notebook contains the code samples found in Chapter 8, Section 1 of [Deep Learning with Python](https://www.manning.com/books/deep-learning-with-python?a_aid=keras&a_bid=76564dff). Note that the original text features far more content, in particular further explanations and figures: in this notebook, you will only find source code and related comments.\n",
    "\n",
    "----\n",
    "\n",
    "[...]\n",
    "\n",
    "## Implementing character-level LSTM text generation\n",
    "\n",
    "\n",
    "Let's put these ideas in practice in a Keras implementation. The first thing we need is a lot of text data that we can use to learn a \n",
    "language model. You could use any sufficiently large text file or set of text files -- Wikipedia, the Lord of the Rings, etc. In this \n",
    "example we will use some of the writings of Nietzsche, the late-19th century German philosopher (translated to English). The language model \n",
    "we will learn will thus be specifically a model of Nietzsche's writing style and topics of choice, rather than a more generic model of the \n",
    "English language."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preparing the data\n",
    "\n",
    "Let's start by downloading the corpus and converting it to lowercase:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Corpus length: 600893\n"
     ]
    }
   ],
   "source": [
    "import keras\n",
    "import numpy as np\n",
    "\n",
    "path = keras.utils.get_file(\n",
    "    'nietzsche.txt',\n",
    "    origin='https://s3.amazonaws.com/text-datasets/nietzsche.txt')\n",
    "text = open(path).read().lower()\n",
    "print('Corpus length:', len(text))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Next, we will extract partially-overlapping sequences of length `maxlen`, one-hot encode them and pack them in a 3D Numpy array `x` of \n",
    "shape `(sequences, maxlen, unique_characters)`. Simultaneously, we prepare a array `y` containing the corresponding targets: the one-hot \n",
    "encoded characters that come right after each extracted sequence."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of sequences: 200278\n",
      "Unique characters: 58\n",
      "Vectorization...\n"
     ]
    }
   ],
   "source": [
    "# Length of extracted character sequences\n",
    "maxlen = 60\n",
    "\n",
    "# We sample a new sequence every `step` characters\n",
    "step = 3\n",
    "\n",
    "# This holds our extracted sequences\n",
    "sentences = []\n",
    "\n",
    "# This holds the targets (the follow-up characters)\n",
    "next_chars = []\n",
    "\n",
    "for i in range(0, len(text) - maxlen, step):\n",
    "    sentences.append(text[i: i + maxlen])\n",
    "    next_chars.append(text[i + maxlen])\n",
    "print('Number of sequences:', len(sentences))\n",
    "\n",
    "# List of unique characters in the corpus\n",
    "chars = sorted(list(set(text)))\n",
    "print('Unique characters:', len(chars))\n",
    "# Dictionary mapping unique characters to their index in `chars`\n",
    "char_indices = dict((char, chars.index(char)) for char in chars)\n",
    "\n",
    "# Next, one-hot encode the characters into binary arrays.\n",
    "print('Vectorization...')\n",
    "x = np.zeros((len(sentences), maxlen, len(chars)), dtype=np.bool)\n",
    "y = np.zeros((len(sentences), len(chars)), dtype=np.bool)\n",
    "for i, sentence in enumerate(sentences):\n",
    "    for t, char in enumerate(sentence):\n",
    "        x[i, t, char_indices[char]] = 1\n",
    "    y[i, char_indices[next_chars[i]]] = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building the network\n",
    "\n",
    "Our network is a single `LSTM` layer followed by a `Dense` classifier and softmax over all possible characters. But let us note that \n",
    "recurrent neural networks are not the only way to do sequence data generation; 1D convnets also have proven extremely successful at it in \n",
    "recent times."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "d:\\Apps\\Python\\Python312\\Lib\\site-packages\\keras\\src\\layers\\rnn\\rnn.py:200: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead.\n",
      "  super().__init__(**kwargs)\n"
     ]
    }
   ],
   "source": [
    "from keras import layers\n",
    "\n",
    "model = keras.models.Sequential()\n",
    "model.add(layers.LSTM(128, input_shape=(maxlen, len(chars))))\n",
    "model.add(layers.Dense(len(chars), activation='softmax'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since our targets are one-hot encoded, we will use `categorical_crossentropy` as the loss to train the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "optimizer = keras.optimizers.RMSprop(learning_rate=0.01)\n",
    "model.compile(loss='categorical_crossentropy', optimizer=optimizer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training the language model and sampling from it\n",
    "\n",
    "\n",
    "Given a trained model and a seed text snippet, we generate new text by repeatedly:\n",
    "\n",
    "* 1) Drawing from the model a probability distribution over the next character given the text available so far\n",
    "* 2) Reweighting the distribution to a certain \"temperature\"\n",
    "* 3) Sampling the next character at random according to the reweighted distribution\n",
    "* 4) Adding the new character at the end of the available text\n",
    "\n",
    "This is the code we use to reweight the original probability distribution coming out of the model, \n",
    "and draw a character index from it (the \"sampling function\"):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def sample(preds, temperature=1.0):\n",
    "    preds = np.asarray(preds).astype('float64')\n",
    "    preds = np.log(preds) / temperature\n",
    "    exp_preds = np.exp(preds)\n",
    "    preds = exp_preds / np.sum(exp_preds)\n",
    "    probas = np.random.multinomial(1, preds, 1)\n",
    "    return np.argmax(probas)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Finally, this is the loop where we repeatedly train and generated text. We start generating text using a range of different temperatures \n",
    "after every epoch. This allows us to see how the generated text evolves as the model starts converging, as well as the impact of \n",
    "temperature in the sampling strategy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 1\n",
      "\u001b[1m1565/1565\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m82s\u001b[0m 52ms/step - loss: 2.3101\n",
      "--- Generating with seed: \" combined with religion itself and regarded as the\n",
      "supreme a\"\n",
      "------ temperature: 0.2\n",
      " combined with religion itself and regarded as the\n",
      "supreme and such and this spirit and sensious intelled the such a condection of the dispanded in the disposited in the great and sensious and and all the despines of the will the deast of the such and of the self-and the will the decisely and of the despines of the great of the will the disposited in the such a prooled the freeds of the dispisious and and be a such a freed and and the all the dispanged and\n",
      "------ temperature: 0.5\n",
      " and and be a such a freed and and the all the dispanged and which is all the greeus the and and this indigious, and every and also the not and a this\n",
      "proves, the right to that the reallous cure as a cholongents--this his not has and like in the inffulice in the regrent and the right which sension in the greatious indering and care its of a great and\n",
      "pladice is its or the such a man it is all the will it\n",
      "is not of the astanded and for his freeds\n",
      "of the ast\n",
      "------ temperature: 1.0\n",
      "will it\n",
      "is not of the astanded and for his freeds\n",
      "of the astadnard yrous\n",
      "he is than\n",
      "heselvent illter.--p and insiged\" a\n",
      "mays which he pucis-intelves coride and wishen dound hiplingme? the rigranity courtion, but moke divenaged a chrissuspanness, ruch, in all befo=ness, stither faith \"tolso itself uns upprandly!\n",
      "\n",
      "2:\n",
      " \n",
      "inccient,\n",
      " part tourastsing meding of hersminall--an acroce.,\n",
      "\n",
      "\n",
      "6盲\n",
      "\n",
      "scount by the himself the cale:, eveslaxitiou hilganicaring, thieke he pr\n",
      "------ temperature: 1.2\n",
      " himself the cale:, eveslaxitiou hilganicaring, thieke he prenopentions buld gownemar,\n",
      "noelde, and oppotent no dasfwindation of e freed of the agall undating\n",
      "mensiout, \n",
      "71!, rom, to dembligilatoation,\n",
      "a chirit custence!\", still 1but\n",
      "his inncoinrridity infeelitite inthis,\n",
      "hincliditf, isseliress which not it.\n",
      " ye stort a whichone our is lighone twere'y, todanded,\n",
      "encumss whee chopendy iv\n",
      "ef wsuczised). which naseaimaniod and anstrencess insislantation of the\n",
      "epoch 2\n",
      "\u001b[1m1565/1565\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m82s\u001b[0m 52ms/step - loss: 1.6652\n",
      "--- Generating with seed: \" idea[16] (as error) is\n",
      "rich in portent, deep, wonderful, ca\"\n",
      "------ temperature: 0.2\n",
      " idea[16] (as error) is\n",
      "rich in portent, deep, wonderful, can the self-such a more possible and sufficient in the subtle and the self-respective in the self-such an into the self-self-such the self-such an into the self-such is the self-such a subtle the self-such an into the self-such an explate the self-shatter of the same the self-such the self-something the self-such an interpretation of the subtle the subtle and the subtle the self-such a more self-de\n",
      "------ temperature: 0.5\n",
      "ubtle the subtle and the subtle the self-such a more self-descibation and the subtle but is the she in order to mentive of the farses of subtle and present of the same it not as a more philosophy of man is the more the subtres and mour decomple and realing to the berenges in the respectives of mind the worts and most is artistific of subtle the distrust of the extent and free of the viluation of the ancess and the more far for of the self that a present wh\n",
      "------ temperature: 1.0\n",
      "he ancess and the more far for of the self that a present which an eave allsispatection. the lifeless are, the master and viows, in the like strize bood winds atheapoved i to be the subjerstiflent. even unneth of the serious and mernessible\n",
      "hoques of litactel from the arrefusions; his adage, hil over selesevence. the  lang! and that the resilt ildisome any among withis, , means and life?-the eardethed butness us conducts of europess of prodic and no snut t\n",
      "------ temperature: 1.2\n",
      "thed butness us conducts of europess of prodic and no snut the imprussion,\n",
      "doghing it, intmen to lasth misunf)ibiling of\n",
      "that\n",
      "t5\n",
      "the surtlef; the rirel: \"wecto it betger\" one\"- seed which that is agailiming \n",
      "or'\n",
      "multhation--flide\n",
      "nitiout:csinol which as it. the ofin of\n",
      "that aoklwabse\n",
      "one\n",
      "the nor strically\n",
      "of the reruded\n",
      "reclofs, and change and unnotoot which\n",
      "still their resirew\n",
      "yeing\n",
      "ex:ment mer.wer the\n",
      "degicouallitye mantabled--whichue \"deally whilh \"idvi\n",
      "epoch 3\n",
      "\u001b[1m1565/1565\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m86s\u001b[0m 55ms/step - loss: 1.5425\n",
      "--- Generating with seed: \"deal of the most world-approving, exuberant, and vivacious m\"\n",
      "------ temperature: 0.2\n",
      "deal of the most world-approving, exuberant, and vivacious moral the spirit of the present the present the spirit of the desires to be any the manking the sense of the sense of the presense the present the sense of the present to the same the man the most manking to the presense the presens of the spirit of the superiority of the present the sense of the present the spirit of the most self-despect to the most desires to be the desires and in the present th\n",
      "------ temperature: 0.5\n",
      " to the most desires to be the desires and in the present the most deplies to respecies and sing to the believe light to unitoments with an exception of the mortanical something something and in the will us and resultion of whom the reason of the contemply and so its its all the philosophing of which the hear to be most origin, the pasiward and the soul which can to generally in\n",
      "some all this is the ranger and also destructions of which a  is to be a prese\n",
      "------ temperature: 1.0\n",
      "he ranger and also destructions of which a  is to be a presens. a philosophy to ye my may the\n",
      "edurable,\n",
      "andiganly for the pay to bleed, profoutners of remain and as the weru and\n",
      "it. the nxward? mest most weaked: it ay\n",
      "brengorary, cree.d in they, an inlinely\n",
      "to let use purtishmanly provinity wastelmenished\"=--his viosome of a her something natic its not the have moppoming divine of voluer, our has arishetycional recains usoutuofs this to'\" only be!ind in an\n",
      "------ temperature: 1.2\n",
      "arishetycional recains usoutuofs this to'\" only be!ind in an agruebo脝ly by to whom ofted the be duetray,--this was and falain; should unwel,\n",
      "predisrcying moral motast mastenation, and\n",
      "perfere, be retroueness\" origin\" more true, to \"in so,\n",
      "pusions. the the way snew not from this firxtce\"by nehequable, say as orignarioui berrearly woman-\"xw.\n",
      "a whomed, obtertable.=--in treed\n",
      "cillons muschel unavej:\n",
      "    man, hiv called values whis deate uftring kendering (beyo\n",
      "epoch 4\n",
      "\u001b[1m1565/1565\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m85s\u001b[0m 54ms/step - loss: 1.4783\n",
      "--- Generating with seed: \"    to spy for you from farthest eagle's view.\n",
      "\n",
      "            \"\n",
      "------ temperature: 0.2\n",
      "    to spy for you from farthest eagle's view.\n",
      "\n",
      "                                                                                                                                                                                                                                                                                                                                                                                                                            \n",
      "------ temperature: 0.5\n",
      "                                                                                                                                                                                                                                                                                                                                                                                                                                                                            \n",
      "------ temperature: 1.0\n",
      "                                                             for the greus all an invension ass it is no himself great which e so pate. the philosophy into low the geniuses-sassing and in the invently ordant, and the certain hac just may plenered\n",
      "they althe eored lies in the\n",
      "resent every\n",
      "proiked and\n",
      "doiss opinion. in the grate the difference same does to lead politaters and at purityly, the foregry in the farthy, feeling tempted, one nay\n",
      "good itself at mea\n",
      "------ temperature: 1.2\n",
      "y in the farthy, feeling tempted, one nay\n",
      "good itself at means, barmary dracis yetvers\n",
      "as\n",
      "at all who -gratelesity; the being effectpisist an, supparations, during whod is deake \"bithingness-mirrist inothe exined.to chart--see injure because makes),\n",
      "whee nourd indisturuty ystehming\" curs\"\n",
      "meanst of grows and the unedich, in all in\n",
      "all uns'oppained\n",
      "litter: durt disturbitas,\n",
      "3pperer, charact mighty, and pieakancy with onessechwaon, prodibl, the belief wild. i\n",
      "epoch 5\n",
      "\u001b[1m1565/1565\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m89s\u001b[0m 57ms/step - loss: 1.4388\n",
      "--- Generating with seed: \"om an anguish with which no other is to be compared.\n",
      "he sees\"\n",
      "------ temperature: 0.2\n",
      "om an anguish with which no other is to be compared.\n",
      "he sees it is a preternations of the spirit in the spirit of the spirit of the spirit of the far and the spirit of the spirit in the spirit of the same the same conserve of the spirit of the present the spirit of the present the spirit is a moral present and the spirit of the spirit of the spirit and soul of the spirit and stronger the spirit of the death of the spirit of the spirit of the spirit in the \n",
      "------ temperature: 0.5\n",
      " the death of the spirit of the spirit of the spirit in the soul, and who always the music, to about and consert the still be appearances of a plato of any actions of the most moralism of the dangerous good and superiority of the has its self-despised to say, and notion of the spirit of the pretest many to any thoughts which is no former the matter in the love of the many be surmine himself and intellectual far of the sanity shates and fundamental the man \n",
      "------ temperature: 1.0\n",
      "tellectual far of the sanity shates and fundamental the man intelleds, for hertherings upon which yead and certain the lasten\" to seated\n",
      "nothing, where man in a develosi in that himself the poterning, for a twicking\n",
      "because tous be proace, a do not when, conserved residetlas incitnestsized wompess\"), necement to the experience hand.?\n",
      "\n",
      "\n",
      "93. preason betraking and appearance betwae' bblook a pinzh predion enough this\n",
      "certaty in the espomes gure sufferigant pa\n",
      "------ temperature: 1.2\n",
      "edion enough this\n",
      "certaty in the espomes gure sufferigant patings things to feared inrite! strong certult, pursive and\n",
      "popert orly reprisis on the but he hidders iful))\"und, indeacy of the vallass, the rules to acquire.\n",
      "\n",
      "a\n",
      " the\n",
      "sphie which\n",
      "discoursely, it animal, ago exerales,\n",
      "ald from excement;--hywhee a\n",
      " t ikating inconvelses of which\n",
      "german speciomifuly in\n",
      "aptermin that hands wil strictom,\n",
      "a hotilt?\n",
      ", -he the brubor,) certionly,\n",
      "sorcest:wardomarism \"hum\n",
      "epoch 6\n",
      "\u001b[1m1565/1565\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m92s\u001b[0m 59ms/step - loss: 1.4057\n",
      "--- Generating with seed: \" and attenuate the will, nothing is\n",
      "so adapted to the spirit\"\n",
      "------ temperature: 0.2\n",
      " and attenuate the will, nothing is\n",
      "so adapted to the spirits of the far as a morality of the soul of the conception of the same and seems the fact so the most as the conduct and also the good and the fatherly the standand the fathers of the states and so may be being and stronger the states of the sense of the fatherly as the state of the futures of the same the sense of the fact which the sense of the states and who has a present the fact of the states a\n",
      "------ temperature: 0.5\n",
      "of the states and who has a present the fact of the states and consequence of schopenhaue in the origin, he befond and faith of the conceale of the fact should has been defilition of them.\n",
      "\n",
      "11t conscience of the present for the among the former such as he belong the as a morality.--and the same the success of your one and we have been something has been the\n",
      "present and the present with the strength man is the bones, there is a more of the belief the intert\n",
      "------ temperature: 1.0\n",
      "h man is the bones, there is a more of the belief the intertric stort, and sted eached him a moral\n",
      "man and that they the attal what, he has\n",
      "beditutineds of humantily\n",
      "statesor pailedly of its grandful as is healted their polity sen taintation,--as this her althe among this hellces: hin times who have newer age of what\n",
      "\"yet we as those the\n",
      "salic people it, everything,, and real to god of--so stupidity overtomest to eqoaded\n",
      "readity, of the\n",
      "sin sy \"hitherto ni\n",
      "------ temperature: 1.2\n",
      "ty overtomest to eqoaded\n",
      "readity, of the\n",
      "sin sy \"hitherto niffuce of\n",
      "a include? s!\n",
      "thing he con cus that howeverphen, rediccriftit something \"a  insty ippassion is exhacisty, is also\n",
      "do-\"naver?\n",
      "where necable it\n",
      "misunt sugble to carmile and retownxtoothing us--they ha if\n",
      "questions; so assvely concerned. for a gover\n",
      "to\n",
      "be zadx--m, only the tendablies of our slaver:\n",
      "it is quite in humanary\n",
      "manifew; \n",
      "oner has it out intellect are cause of all theith himself, a\n",
      "epoch 7\n",
      "\u001b[1m1565/1565\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m85s\u001b[0m 54ms/step - loss: 1.3964\n",
      "--- Generating with seed: \"y)\"--he had said, or at least meant to\n",
      "say. but, is that--an\"\n",
      "------ temperature: 0.2\n",
      "y)\"--he had said, or at least meant to\n",
      "say. but, is that--and who are always and the strength and the sure of a society and the and the soul of the according to the such and the condition of the spirit and some things of the far as a properting the and the same of the words of the present of the spirit and also its are before and the spirit and such and the words of the and the same and the same and discoverity and the strict science the spirit and the sur\n",
      "------ temperature: 0.5\n",
      "nd discoverity and the strict science the spirit and the sure, and more and connorned the refine and such a personal and discoverience of the desires and which still the varied and nevertheless a sympathy and simplical spirituality to see the will to see one will to \"mensistord of the will of the nature and from the all their will and some deward of the such and cannotion are for the same the individe the belief in the work of the instance the simplical an\n",
      "------ temperature: 1.0\n",
      "vide the belief in the work of the instance the simplical and immoians without powerfand.\n",
      "\n",
      "\n",
      "\n",
      "=zronness. the will cameabl causaristic should be with a low--here more powerture\n",
      "the spirit.\"\n",
      "\n",
      "13盲. we have\n",
      "sided is\n",
      "provolited philosophy its virtue of will require the orr\n",
      "saino dividuans, as the   from honeshess whom a\n",
      "most rare femble amsely, were percimsen one wcood hence of mankind-\"indeceivation of eximmter. but whoevever dipantable looked that as dow--the \n",
      "------ temperature: 1.2\n",
      " eximmter. but whoevever dipantable looked that as dow--the coivellesses.\n",
      "indelf, abidezable on all thought:\n",
      "out of hbight\n",
      "its\n",
      "own ethicous\"s,\n",
      "strtitu fol--yece mo:sard) and or leadient?\n",
      "\n",
      "yer or keep; which who are of the our graut\" mastiletabiess of\n",
      "hibtf the indiffious pride a tempts\n",
      "thelly supporiencements, which\n",
      "are daked\n",
      "to extating man\n",
      "willed,\n",
      "who prolous\n",
      "for which ?as anothts of when\n",
      "wereither, as\n",
      "a culture\n",
      "submit destrible them oneself believed im \n",
      "epoch 8\n",
      "\u001b[1m1565/1565\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m89s\u001b[0m 57ms/step - loss: 1.3775\n",
      "--- Generating with seed: \"nd trouble ourselves to gain health). we\n",
      "conclude from analo\"\n",
      "------ temperature: 0.2\n",
      "nd trouble ourselves to gain health). we\n",
      "conclude from analopes and and the instinction the spiritual and in the same the spiritual presence of the spiritual and sure, the same the spiritual and consider the spiritual and in the strength and with the will to be the same the same the spiritual instinction of the explaine and interpretation of the presention with the standing the spiritual explaine and such and in the sense of the spiritual and even and such\n",
      "------ temperature: 0.5\n",
      "and such and in the sense of the spiritual and even and such and present work of the enthulation. in it is not in even of the good and better with a spiritual instroys and with a noble and to be any possible and religious inclines of any one should there are in the mortary and pressant of the individuality of the lighterning of will in even of the rest in disconception in the english of point\n",
      "now to say that the spiritual principles of the surfict and inve\n",
      "------ temperature: 1.0\n",
      "to say that the spiritual principles of the surfict and inventine will: they would fimsel suppores,\n",
      "his corruptive in seccess to cragicibis intention and woman, proviction of\n",
      "the genarost in the illens give eliminal practicically very cleading of this mad of decraine of our lighth the higher\n",
      "work than endriver one--is, inevirtory from pride with \"accicisis? with regarding view their being and ascentant indrihep\n",
      " the youh. they to antir\n",
      "christian\n",
      "muld -a\n",
      "li\n",
      "------ temperature: 1.2\n",
      "ntant indrihep\n",
      " the youh. they to antir\n",
      "christian\n",
      "muld -a\n",
      "lide our r'vertit茅dme differed does\n",
      "sjemority and abssedable instinctide tod,\n",
      "whoegret. limiany pompassiur?--in who be licters, ranger, on the intredly ide, experiegy where\n",
      "lacy be, even if itjers, seed, inducer more fundamentictively cannoth in\n",
      "h\n",
      "justified wpeny, for that, music vorty, but withoot in germ)fle\n",
      "and dution-de;righthafe]?fither to agrivated\n",
      "tricin und heine\n",
      "of lightesd thut, invirine w\n",
      "epoch 9\n",
      "\u001b[1m1565/1565\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m96s\u001b[0m 61ms/step - loss: 1.3639\n",
      "--- Generating with seed: \"re made and authorities prescribed. but far more potent\n",
      "is t\"\n",
      "------ temperature: 0.2\n",
      "re made and authorities prescribed. but far more potent\n",
      "is the state of the state of the strong and the strength of the same that the sense of the such a man who is a prisile and the strong and the strong and the state of the strength, and the man who is the strong and the superiority of the state of the superiority and the strong and the strong and that is the stand is the stand and and the sense of the strong and the world with the strong the state of th\n",
      "------ temperature: 0.5\n",
      " of the strong and the world with the strong the state of the cause in the struck constinution that a constinut of a being \"                                                                                                                                                                                                                                                                                                                                                \n",
      "------ temperature: 1.0\n",
      "                                                                         some deverant. ihrell-let subsequ]nts and of invent) the sketricious,\n",
      "grow will, as the moral conscier from \"sinfest they can we believe the and free listen disprisal know never always be averapted haff the instruce intimitances also men, thou that has been to mankinds, something man\n",
      "confounity mead and evilse it tragifiorative invent still outsistic, which there and psychologicis its\n",
      "------ temperature: 1.2\n",
      "ve invent still outsistic, which there and psychologicis its forming to geterate and relationsly not exist.=-the arisocken, atly\n",
      "long-the conduct, it\n",
      "is althys of indquite man, will which was coulfly--which his perzijsd, nowady enrave--the voluntless\" ware\" sy thus\n",
      "goba, hundp.\"\n",
      "\n",
      "86w[y in arrelw: ye mamsest hivily disguise fluccori),, their\n",
      "fact promcer6ly. that is crimis, a pergectpotion awactions-\"eleval sacrifite and in ihestly e\n",
      "acceltonike of the numb\n",
      "epoch 10\n",
      "\u001b[1m1565/1565\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m83s\u001b[0m 53ms/step - loss: 1.3539\n",
      "--- Generating with seed: \"eving christian\n",
      "learns from his church). in fact, conformabl\"\n",
      "------ temperature: 0.2\n",
      "eving christian\n",
      "learns from his church). in fact, conformable instinct of the same such an accordinal and characters of the contrary and something which all the bearing and conception of the same the man they have a the same such an accordinal and promperty of the man they have a conscience of the spiritual and such an interest and all the really and acts and they are all the existence of all the same the same and the man and the same the same the same ref\n",
      "------ temperature: 0.5\n",
      "same the same and the man and the same the same the same reflect in the same fact of the such and all the same personalism, on the man and the same the laws of itself that an inventres is all the man was the spirit and can for only constantly man has not been according to the spiritual forms the same things and their serity of the animal and similar works of the such and soul and all they have in all the hast in the interes and the interpretation of the sa\n",
      "------ temperature: 1.0\n",
      "all the hast in the interes and the interpretation of the same, or helperorical\n",
      "potishing deternating\n",
      "of of still great more foregranting are eativing, extermons false for ever no\n",
      "much are in \"evilting\" evil long of man\"ro qualiciable,\" or workery, man is higher sp, to some bood of historical emonix on this being our literatic in their other of the solitude. they are desirations of conscience, short, have\n",
      "harn only excentances, however, however, and means \n",
      "------ temperature: 1.2\n",
      "rt, have\n",
      "harn only excentances, however, however, and means with xartain dangers, and\n",
      "unmjigatesly shorm of many re:initement, susce on ta fool4 for a     under\" all\n",
      "the attempting and expreceed by hating anticious in, chearly fices of such alt it?\n",
      "act nothing i alfached deceived fanchessagual nature eye!; let us not\n",
      "to sinfatigatic is estent of faving posed only gnowlenstrayking, is unpoles lacking, events, as\n",
      "moderines of god-factory of an thingss-schole\n",
      "epoch 11\n",
      "\u001b[1m1565/1565\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m87s\u001b[0m 56ms/step - loss: 1.3437\n",
      "--- Generating with seed: \"ree spirit is bothered with mere things--and how many\n",
      "things\"\n",
      "------ temperature: 0.2\n",
      "ree spirit is bothered with mere things--and how many\n",
      "things and same the sense of the same things and for the sense that the sense of such a man which the same interpretation of the same and conduct and self-recertain do not be a good and more own things and saint and life and saint of the sense of the sense of the same and self time a man all the same and same that the same and found that the standard of the sense of the produce in the same and desire an\n",
      "------ temperature: 0.5\n",
      "andard of the sense of the produce in the same and desire and for the light and more person of his does to be be long as a soul. a constitute that which the present the invention and pretendent of the inclination is a man the soul is in the burden and what is not be read in the interpretation of the problegily of the present to present the indued, and and lack in the science in the developed that he in the scholets which man such and the subtre to the sign\n",
      "------ temperature: 1.0\n",
      "he in the scholets which man such and the subtre to the significant which\n",
      "good poor and\n",
      "\"prulations. it is europeaus fest] all he thing make willual man instinct, and invention\n",
      "of the transfearst_ the thing, philosopho bad tradication back: man whwile al enthumate in disguises sens this\n",
      "care be war the means of fument of shembble, in sensity will be himself, it is it selfod; as the mask,\n",
      "which a mor! to that a time our man,\n",
      "e\n",
      "hearts\n",
      "and it being it reserva\n",
      "------ temperature: 1.2\n",
      "or! to that a time our man,\n",
      "e\n",
      "hearts\n",
      "and it being it reservation thhiimed, more gar god that how roince and sinderles oveing. a voord of\n",
      "values treamble lami-a prieks aqual wiintne skeptive docush in rugh with\n",
      "that\n",
      "oy\n",
      "the vapk aw), with the weykers that eor perslus man.\n",
      "\n",
      "\n",
      "eete inaterent to unfectly dianeme of my cove. that\n",
      "gold-ornatung for their feigund gulanety or said, my of religidical notion imsagultserves; i rather-e-ignemony the extent, spea our nec\n",
      "epoch 12\n",
      "\u001b[1m1565/1565\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m80s\u001b[0m 51ms/step - loss: 1.3353\n",
      "--- Generating with seed: \"indolence. next he seeks the battle and extinguishes it with\"\n",
      "------ temperature: 0.2\n",
      "indolence. next he seeks the battle and extinguishes it with the spirit and the spirit and the strong the fact the spirit and the sublimering the command in the same the such an explosarily in the spirit and the strong to the such an interest the latter to be the spirit and the such an and such and in the spirit and profoundly in the same the sublimes the standard of the same the sublimes of the same the profoundly in such and the same the spirit and stron\n",
      "------ temperature: 0.5\n",
      "ame the profoundly in such and the same the spirit and strong and who really passion of a child himself, and in the earth. so this idea which we free spirit is not the distinction, and the most possess whose sharly divine form of the part of the sense of the free spirits of elliguan and most be and his powerful and\n",
      "responsibility of his heart believe the free spirit and successful and the great the profoundly death the whoe, as a not the noble anything who\n",
      "------ temperature: 1.0\n",
      "e profoundly death the whoe, as a not the noble anything who shouth with parks the getide. this spirit from the free mbo bech, atteins wellas.\"--i do sang\n",
      "to given--a whole and lome\n",
      "the sensutourd\n",
      "in the boodistic. i deporation,; and he ready an interest bangness,\n",
      "had other dungeong these morality and bysurs and ethicly in himself, he psys profound, the mast even all nothing hlies steaves, pompoms this charms which very criticuses or\n",
      "womancorize a probe do\n",
      "------ temperature: 1.2\n",
      " this charms which very criticuses or\n",
      "womancorize a probe dosives so, upoblished\n",
      "cipliza-stin co\n",
      "rule of one's\n",
      "truth.\" they delights \"natustly\n",
      "\"soul mattery, \"men.i stil culture: let us englishd as, ifhunged.\n",
      "\n",
      "\n",
      "75e\n",
      "nous\n",
      "has, hyet, is bay in isontuness the hoocour dis\" something which throughd.\n",
      "\n",
      "15盲. plibakiny, were isselvt\") rutitadpest be?\"hd-invent into god radu\n",
      "have have pride,\n",
      "sircestack much but by\n",
      "truththyly is a mlinfound, an even spirits belongee, \n",
      "epoch 13\n",
      "\u001b[1m1565/1565\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m84s\u001b[0m 53ms/step - loss: 1.3252\n",
      "--- Generating with seed: \"s the brain immediately conjures up a\n",
      "medley of impressions \"\n",
      "------ temperature: 0.2\n",
      "s the brain immediately conjures up a\n",
      "medley of impressions and interpretation of the words of the same that the same things and the protection of the individually any men and individual in the philosophy--and the existed to see the fact, and the most point of the sense of the same the soul of the most protection and individually an any intermine the fact of the sense of the words and soul and all the sense of the sense of the same them and all the most ha\n",
      "------ temperature: 0.5\n",
      " the sense of the sense of the same them and all the most hard whole never strifted beaded and in the existed itself with the most and be taken to see the heart to so partly he so its determined and has and ablothing with the most evil as a morality of developed and conservation of the same history, and the words in the same time all them in all them all the distings all advantage, the first the form through the form of the latters the readors all the grea\n",
      "------ temperature: 1.0\n",
      "orm through the form of the latters the readors all the greater sharlymenes. it is that called not an expressions that a disturpity, we sey oundifound misarvification of life ald br: as an oblingment of done by, woman; four that swele of\n",
      "the modistux in soncition is the animal singums of the wast en\n",
      "hidsee, lectification that is, any he shults\n",
      "a formful leckaties, as acrothous even motivess.\n",
      "xeee put here who\n",
      "conclusitics and to the intumpatic\n",
      "command,\n",
      "ama\n",
      "------ temperature: 1.2\n",
      "put here who\n",
      "conclusitics and to the intumpatic\n",
      "command,\n",
      "amasty asua, certain disclived asirroin\n",
      "too things\n",
      "extent hand, he speak: many a well, in the apopious: you causing p\"judical my humanity, to juisu flash domain to be our p: aregenced-the\n",
      "experiteness,\n",
      "i create-pading\n",
      "oare ethic intoeectuous sensationed with la folly sufficionably\n",
      "no\n",
      "clined curse means with noural refugement him by la recoloci, the fawards his folloty it might him is\n",
      "grateened by sai\n",
      "epoch 14\n",
      "\u001b[1m1565/1565\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m92s\u001b[0m 59ms/step - loss: 1.3231\n",
      "--- Generating with seed: \"e other hand, it is known by the experience of breeders that\"\n",
      "------ temperature: 0.2\n",
      "e other hand, it is known by the experience of breeders that is the same and sufficient and sufficient for the same and sure, and in the same and sufficient and sufficient and such a state of the most standomen of the facts and the same and sufficient and sufficient to the entire and such a power and \"the same and sure, the same in the most proposition of the proper and the same in the same in the same experience of the philosophers in the states and suffi\n",
      "------ temperature: 0.5\n",
      " same experience of the philosophers in the states and sufficient with the conscience\n",
      "in the facts in the most the rest of the spirit and disposition to series perfucted and similation and superiation of fact that is not the disposing to the est from the most from the powerful to the same become the same man who has interesting to him, and with a repusiod in the souls the condition of an inverse to the senses power, and accordingly of the power, and in him\n",
      "------ temperature: 1.0\n",
      "o the senses power, and accordingly of the power, and in himself, the made for the depth! if the sensentic inour pate. began to goes to conscholy ease pated, mucicat remay momen, and that laugh what the old conditions of old\n",
      "doces acts an when into made originated in\n",
      "order him for himself laws.=--afto-buence of the instrument.\n",
      "\n",
      "    read the else and donfered\n",
      "in ay\n",
      "entailirable unnectives one much closely he on the populaols. for indeed enjoys fimally spirs\n",
      "------ temperature: 1.2\n",
      "closely he on the populaols. for indeed enjoys fimally spirstod.\"l us. if weken dividual once give plaes flakble\n",
      "he it, prect\n",
      "at the spcritombous faib\n",
      "and self-geing thrutal youth and\n",
      "himselfceed national and it. infloed get and wild and ny arconceptalivating, certain symptom, lra-gotwhiel throod of a very elour croureing, artain purely believe closeg\n",
      "and). the german respectable hamments, the \n",
      "time\n",
      "aradir frimgration remaris of \"raling by reallyfited--exp\n",
      "epoch 15\n",
      "\u001b[1m1565/1565\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m95s\u001b[0m 61ms/step - loss: 1.3189\n",
      "--- Generating with seed: \"pparently opposed impulse of the spirit, a suddenly adopted \"\n",
      "------ temperature: 0.2\n",
      "pparently opposed impulse of the spirit, a suddenly adopted and desires and the spirit and consequence of the another in the such a stronger the sense of the fact the successful and development of the successful things that the philosophy of the moral successful to the existence of a more instincts of the same the successful and desires and development of the consequence of the successful instincts, and always the successful things the consequence of the s\n",
      "------ temperature: 0.5\n",
      "s, and always the successful things the consequence of the sense of consession to look evis in the tenties and the far as it is a present to a single of the religious through the endured every one morality of the present deterience of a more history of men that their are sphich and precisely as in which is the devil bet. as the community in the eyes and expeded, and have not eneged the more possible and devil\n",
      "self-strongeral metaphysical perfucts of the su\n",
      "------ temperature: 1.0\n",
      "le and devil\n",
      "self-strongeral metaphysical perfucts of the suspiciation in inetious to the freatom of all sufferently artulan lies\n",
      "\"whom in the hetpeus justicate as him stepter, and put as a look evil--in all dosmen or indeed a vait regards the will of\n",
      "acts may\n",
      "be precaitary\n",
      "some the casis of,\n",
      "evilpoistic, and stood of physial, good will in the schopenhauer's from\n",
      "himself in\n",
      "that\n",
      "loy in all on inspirity of this whole thinking her with under nestive as\n",
      "looka\n",
      "------ temperature: 1.2\n",
      "irity of this whole thinking her with under nestive as\n",
      "lookasphein, over.\n",
      "\n",
      "s] ho looked to theirocous aid prisis--co e, if does thistell believe which should isundestestion--i\" bejod, grown uputher \"tenmine knowledge;s insens\n",
      "be all difficual.\n",
      "\n",
      "\n",
      "1茅      just as ither by脝\"yous limitation much is thereto,\n",
      "\"infarming--or\n",
      "therein\n",
      "this\n",
      "principally, to la,\n",
      "coming herith, who ist there handfecato host which like in the\n",
      "strength one one another him. in ny lone\n",
      "-in\n",
      "epoch 16\n",
      "\u001b[1m1565/1565\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m92s\u001b[0m 59ms/step - loss: 1.3058\n",
      "--- Generating with seed: \"nce in the exercise of\n",
      "authority--as a bond which binds rule\"\n",
      "------ temperature: 0.2\n",
      "nce in the exercise of\n",
      "authority--as a bond which binds rule of the same things and the same the same things and the such conceals to the same things and self-conformed to be a morality of the such\n",
      "conscience of the superstitition of the superiority of the conventing the sense of the such personal contempt the same things and present to perceiving the strong and present to the same such a man and some the superiorite and all the superiorite of the such\n",
      "clg\n",
      "------ temperature: 0.5\n",
      "some the superiorite and all the superiorite of the such\n",
      "clgress and well experience and counter that it oughty and the causing, that is the secretly and all\n",
      "the harding, and great for in which is all the incinder good and such a philosophical being and the soul, who would cannot all the possible, a philosophical morality of which makes of a possessioning of the freeks the such\n",
      "clgress in the exceptions which has not it is the sense of the superiorite and \n",
      "------ temperature: 1.0\n",
      "ptions which has not it is the sense of the superiorite and logical looks of the truth.'ptenessly the divines to\n",
      "aimors out; something all truto is flast we not a propertical and nourishely not\n",
      "then our lieh happiness, all unlion, and present the ghren nowadays?\n",
      "\n",
      "o3. merely he wishern and won no the\n",
      "naturely lest itself. this develop lives he\n",
      "oughts time. away? do not does no making, all these selfished if its philosopher were to immediating ill goble\"--th\n",
      "------ temperature: 1.2\n",
      "fished if its philosopher were to immediating ill goble\"--the vaeal recightion? a verwardor becemel feelish extention of people that \n",
      "keedingly the\n",
      "sycessism i sunder\"\n",
      "pet itsidity. \n",
      "\n",
      "\n",
      "on christies to guilt,ew :\n",
      "\n",
      "\n",
      "a preciselyed to be\n",
      "color, in\n",
      "the rights aditulens-portwhen thingss\n",
      "in the chumalacl\". enjoyment\n",
      "of fasuly, as\n",
      "sawarfers of mentimes as at eveckfulness in revournan to\n",
      "leading, bood that secretly, the nownness\n",
      "we must look in any ef lesttigene)s.\n",
      "epoch 17\n",
      "\u001b[1m1565/1565\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m83s\u001b[0m 53ms/step - loss: 1.3056\n",
      "--- Generating with seed: \"he aesthetic\n",
      "feeling is decided, owing to the great number o\"\n",
      "------ temperature: 0.2\n",
      "he aesthetic\n",
      "feeling is decided, owing to the great number of the profoundly and the strong and the same and the problem of the properful to be the sufferings of the sure, and with the problem of the suffers of the strength of the sure the suffers of the profound the stranges that the explanations of the sense of the sure, the problem of the strength of the same and also the problemary and suffers of the superiority of the world of the sublige of the sense\n",
      "------ temperature: 0.5\n",
      " of the superiority of the world of the sublige of the senses in the sume formuling that the fargour explanation of the senses with the same, and\n",
      "the young himself as the ention which has to no derreater from the heartion of the strength. and who were all and in it is not deceive something the instinct of the proud to his own satisfaction that the fact the temporarions of any explanation of philosophers of the infinite the radess of and consequences, and i\n",
      "------ temperature: 1.0\n",
      "ophers of the infinite the radess of and consequences, and it as have not inxicts man, the testrain. habnish man, and must have evisted see has manvise; really to imposes i doc--truth show from faver, with the danger;\n",
      "so perhaps proposiusle, over some taste of \n",
      "sharty somech are gain of which keeps with the \"clinate over of\n",
      "bravelty is alsoan this\n",
      "curressing\n",
      "when have no attaneficing are mani\" any one still bring with my them. if the prognusisarfing presul\n",
      "------ temperature: 1.2\n",
      " one still bring with my them. if the prognusisarfing presuld and full drawss something grautorts truous scfuet tradition of the': \"more moor\n",
      "schopen, that durch, at the words long results of sick orrail, grety in ita hopmess (we to be a too\n",
      "tun nothering not hack; that this, weuponessy, and\n",
      "wond diffation see hims--became\n",
      "prentra\n",
      "responsible from the towards; must designate thans press of sen from themselves. dread\n",
      "just\n",
      "fact, and also far conscience of re\n",
      "epoch 18\n",
      "\u001b[1m 976/1565\u001b[0m \u001b[32m━━━━━━━━━━━━\u001b[0m\u001b[37m━━━━━━━━\u001b[0m \u001b[1m31s\u001b[0m 53ms/step - loss: 1.2912"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[31m---------------------------------------------------------------------------\u001b[39m",
      "\u001b[31mKeyboardInterrupt\u001b[39m                         Traceback (most recent call last)",
      "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[12]\u001b[39m\u001b[32m, line 7\u001b[39m\n\u001b[32m      5\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[33m'\u001b[39m\u001b[33mepoch\u001b[39m\u001b[33m'\u001b[39m, epoch)\n\u001b[32m      6\u001b[39m \u001b[38;5;66;03m# Fit the model for 1 epoch on the available training data\u001b[39;00m\n\u001b[32m----> \u001b[39m\u001b[32m7\u001b[39m \u001b[43mmodel\u001b[49m\u001b[43m.\u001b[49m\u001b[43mfit\u001b[49m\u001b[43m(\u001b[49m\u001b[43mx\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43my\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m      8\u001b[39m \u001b[43m          \u001b[49m\u001b[43mbatch_size\u001b[49m\u001b[43m=\u001b[49m\u001b[32;43m128\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[32m      9\u001b[39m \u001b[43m          \u001b[49m\u001b[43mepochs\u001b[49m\u001b[43m=\u001b[49m\u001b[32;43m1\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[32m     11\u001b[39m \u001b[38;5;66;03m# Select a text seed at random\u001b[39;00m\n\u001b[32m     12\u001b[39m start_index = random.randint(\u001b[32m0\u001b[39m, \u001b[38;5;28mlen\u001b[39m(text) - maxlen - \u001b[32m1\u001b[39m)\n",
      "\u001b[36mFile \u001b[39m\u001b[32md:\\Apps\\Python\\Python312\\Lib\\site-packages\\keras\\src\\utils\\traceback_utils.py:117\u001b[39m, in \u001b[36mfilter_traceback.<locals>.error_handler\u001b[39m\u001b[34m(*args, **kwargs)\u001b[39m\n\u001b[32m    115\u001b[39m filtered_tb = \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[32m    116\u001b[39m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[32m--> \u001b[39m\u001b[32m117\u001b[39m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfn\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m    118\u001b[39m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[32m    119\u001b[39m     filtered_tb = _process_traceback_frames(e.__traceback__)\n",
      "\u001b[36mFile \u001b[39m\u001b[32md:\\Apps\\Python\\Python312\\Lib\\site-packages\\keras\\src\\backend\\tensorflow\\trainer.py:371\u001b[39m, in \u001b[36mTensorFlowTrainer.fit\u001b[39m\u001b[34m(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq)\u001b[39m\n\u001b[32m    369\u001b[39m \u001b[38;5;28;01mfor\u001b[39;00m step, iterator \u001b[38;5;129;01min\u001b[39;00m epoch_iterator:\n\u001b[32m    370\u001b[39m     callbacks.on_train_batch_begin(step)\n\u001b[32m--> \u001b[39m\u001b[32m371\u001b[39m     logs = \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mtrain_function\u001b[49m\u001b[43m(\u001b[49m\u001b[43miterator\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m    372\u001b[39m     callbacks.on_train_batch_end(step, logs)\n\u001b[32m    373\u001b[39m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m.stop_training:\n",
      "\u001b[36mFile \u001b[39m\u001b[32md:\\Apps\\Python\\Python312\\Lib\\site-packages\\keras\\src\\backend\\tensorflow\\trainer.py:219\u001b[39m, in \u001b[36mTensorFlowTrainer._make_function.<locals>.function\u001b[39m\u001b[34m(iterator)\u001b[39m\n\u001b[32m    215\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mfunction\u001b[39m(iterator):\n\u001b[32m    216\u001b[39m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(\n\u001b[32m    217\u001b[39m         iterator, (tf.data.Iterator, tf.distribute.DistributedIterator)\n\u001b[32m    218\u001b[39m     ):\n\u001b[32m--> \u001b[39m\u001b[32m219\u001b[39m         opt_outputs = \u001b[43mmulti_step_on_iterator\u001b[49m\u001b[43m(\u001b[49m\u001b[43miterator\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m    220\u001b[39m         \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m opt_outputs.has_value():\n\u001b[32m    221\u001b[39m             \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mStopIteration\u001b[39;00m\n",
      "\u001b[36mFile \u001b[39m\u001b[32md:\\Apps\\Python\\Python312\\Lib\\site-packages\\tensorflow\\python\\util\\traceback_utils.py:150\u001b[39m, in \u001b[36mfilter_traceback.<locals>.error_handler\u001b[39m\u001b[34m(*args, **kwargs)\u001b[39m\n\u001b[32m    148\u001b[39m filtered_tb = \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[32m    149\u001b[39m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[32m--> \u001b[39m\u001b[32m150\u001b[39m   \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfn\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m    151\u001b[39m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[32m    152\u001b[39m   filtered_tb = _process_traceback_frames(e.__traceback__)\n",
      "\u001b[36mFile \u001b[39m\u001b[32md:\\Apps\\Python\\Python312\\Lib\\site-packages\\tensorflow\\python\\eager\\polymorphic_function\\polymorphic_function.py:833\u001b[39m, in \u001b[36mFunction.__call__\u001b[39m\u001b[34m(self, *args, **kwds)\u001b[39m\n\u001b[32m    830\u001b[39m compiler = \u001b[33m\"\u001b[39m\u001b[33mxla\u001b[39m\u001b[33m\"\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m._jit_compile \u001b[38;5;28;01melse\u001b[39;00m \u001b[33m\"\u001b[39m\u001b[33mnonXla\u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m    832\u001b[39m \u001b[38;5;28;01mwith\u001b[39;00m OptionalXlaContext(\u001b[38;5;28mself\u001b[39m._jit_compile):\n\u001b[32m--> \u001b[39m\u001b[32m833\u001b[39m   result = \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_call\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwds\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m    835\u001b[39m new_tracing_count = \u001b[38;5;28mself\u001b[39m.experimental_get_tracing_count()\n\u001b[32m    836\u001b[39m without_tracing = (tracing_count == new_tracing_count)\n",
      "\u001b[36mFile \u001b[39m\u001b[32md:\\Apps\\Python\\Python312\\Lib\\site-packages\\tensorflow\\python\\eager\\polymorphic_function\\polymorphic_function.py:878\u001b[39m, in \u001b[36mFunction._call\u001b[39m\u001b[34m(self, *args, **kwds)\u001b[39m\n\u001b[32m    875\u001b[39m \u001b[38;5;28mself\u001b[39m._lock.release()\n\u001b[32m    876\u001b[39m \u001b[38;5;66;03m# In this case we have not created variables on the first call. So we can\u001b[39;00m\n\u001b[32m    877\u001b[39m \u001b[38;5;66;03m# run the first trace but we should fail if variables are created.\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m878\u001b[39m results = \u001b[43mtracing_compilation\u001b[49m\u001b[43m.\u001b[49m\u001b[43mcall_function\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m    879\u001b[39m \u001b[43m    \u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mkwds\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_variable_creation_config\u001b[49m\n\u001b[32m    880\u001b[39m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m    881\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m._created_variables:\n\u001b[32m    882\u001b[39m   \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\u001b[33m\"\u001b[39m\u001b[33mCreating variables on a non-first call to a function\u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m    883\u001b[39m                    \u001b[33m\"\u001b[39m\u001b[33m decorated with tf.function.\u001b[39m\u001b[33m\"\u001b[39m)\n",
      "\u001b[36mFile \u001b[39m\u001b[32md:\\Apps\\Python\\Python312\\Lib\\site-packages\\tensorflow\\python\\eager\\polymorphic_function\\tracing_compilation.py:139\u001b[39m, in \u001b[36mcall_function\u001b[39m\u001b[34m(args, kwargs, tracing_options)\u001b[39m\n\u001b[32m    137\u001b[39m bound_args = function.function_type.bind(*args, **kwargs)\n\u001b[32m    138\u001b[39m flat_inputs = function.function_type.unpack_inputs(bound_args)\n\u001b[32m--> \u001b[39m\u001b[32m139\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfunction\u001b[49m\u001b[43m.\u001b[49m\u001b[43m_call_flat\u001b[49m\u001b[43m(\u001b[49m\u001b[43m  \u001b[49m\u001b[38;5;66;43;03m# pylint: disable=protected-access\u001b[39;49;00m\n\u001b[32m    140\u001b[39m \u001b[43m    \u001b[49m\u001b[43mflat_inputs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcaptured_inputs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mfunction\u001b[49m\u001b[43m.\u001b[49m\u001b[43mcaptured_inputs\u001b[49m\n\u001b[32m    141\u001b[39m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n",
      "\u001b[36mFile \u001b[39m\u001b[32md:\\Apps\\Python\\Python312\\Lib\\site-packages\\tensorflow\\python\\eager\\polymorphic_function\\concrete_function.py:1322\u001b[39m, in \u001b[36mConcreteFunction._call_flat\u001b[39m\u001b[34m(self, tensor_inputs, captured_inputs)\u001b[39m\n\u001b[32m   1318\u001b[39m possible_gradient_type = gradients_util.PossibleTapeGradientTypes(args)\n\u001b[32m   1319\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m (possible_gradient_type == gradients_util.POSSIBLE_GRADIENT_TYPES_NONE\n\u001b[32m   1320\u001b[39m     \u001b[38;5;129;01mand\u001b[39;00m executing_eagerly):\n\u001b[32m   1321\u001b[39m   \u001b[38;5;66;03m# No tape is watching; skip to running the function.\u001b[39;00m\n\u001b[32m-> \u001b[39m\u001b[32m1322\u001b[39m   \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_inference_function\u001b[49m\u001b[43m.\u001b[49m\u001b[43mcall_preflattened\u001b[49m\u001b[43m(\u001b[49m\u001b[43margs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m   1323\u001b[39m forward_backward = \u001b[38;5;28mself\u001b[39m._select_forward_and_backward_functions(\n\u001b[32m   1324\u001b[39m     args,\n\u001b[32m   1325\u001b[39m     possible_gradient_type,\n\u001b[32m   1326\u001b[39m     executing_eagerly)\n\u001b[32m   1327\u001b[39m forward_function, args_with_tangents = forward_backward.forward()\n",
      "\u001b[36mFile \u001b[39m\u001b[32md:\\Apps\\Python\\Python312\\Lib\\site-packages\\tensorflow\\python\\eager\\polymorphic_function\\atomic_function.py:216\u001b[39m, in \u001b[36mAtomicFunction.call_preflattened\u001b[39m\u001b[34m(self, args)\u001b[39m\n\u001b[32m    214\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mcall_preflattened\u001b[39m(\u001b[38;5;28mself\u001b[39m, args: Sequence[core.Tensor]) -> Any:\n\u001b[32m    215\u001b[39m \u001b[38;5;250m  \u001b[39m\u001b[33;03m\"\"\"Calls with flattened tensor inputs and returns the structured output.\"\"\"\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m216\u001b[39m   flat_outputs = \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mcall_flat\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43margs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m    217\u001b[39m   \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m.function_type.pack_output(flat_outputs)\n",
      "\u001b[36mFile \u001b[39m\u001b[32md:\\Apps\\Python\\Python312\\Lib\\site-packages\\tensorflow\\python\\eager\\polymorphic_function\\atomic_function.py:251\u001b[39m, in \u001b[36mAtomicFunction.call_flat\u001b[39m\u001b[34m(self, *args)\u001b[39m\n\u001b[32m    249\u001b[39m \u001b[38;5;28;01mwith\u001b[39;00m record.stop_recording():\n\u001b[32m    250\u001b[39m   \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m._bound_context.executing_eagerly():\n\u001b[32m--> \u001b[39m\u001b[32m251\u001b[39m     outputs = \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_bound_context\u001b[49m\u001b[43m.\u001b[49m\u001b[43mcall_function\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m    252\u001b[39m \u001b[43m        \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mname\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m    253\u001b[39m \u001b[43m        \u001b[49m\u001b[38;5;28;43mlist\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43margs\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m    254\u001b[39m \u001b[43m        \u001b[49m\u001b[38;5;28;43mlen\u001b[39;49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mfunction_type\u001b[49m\u001b[43m.\u001b[49m\u001b[43mflat_outputs\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m    255\u001b[39m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m    256\u001b[39m   \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m    257\u001b[39m     outputs = make_call_op_in_graph(\n\u001b[32m    258\u001b[39m         \u001b[38;5;28mself\u001b[39m,\n\u001b[32m    259\u001b[39m         \u001b[38;5;28mlist\u001b[39m(args),\n\u001b[32m    260\u001b[39m         \u001b[38;5;28mself\u001b[39m._bound_context.function_call_options.as_attrs(),\n\u001b[32m    261\u001b[39m     )\n",
      "\u001b[36mFile \u001b[39m\u001b[32md:\\Apps\\Python\\Python312\\Lib\\site-packages\\tensorflow\\python\\eager\\context.py:1688\u001b[39m, in \u001b[36mContext.call_function\u001b[39m\u001b[34m(self, name, tensor_inputs, num_outputs)\u001b[39m\n\u001b[32m   1686\u001b[39m cancellation_context = cancellation.context()\n\u001b[32m   1687\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m cancellation_context \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m-> \u001b[39m\u001b[32m1688\u001b[39m   outputs = \u001b[43mexecute\u001b[49m\u001b[43m.\u001b[49m\u001b[43mexecute\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m   1689\u001b[39m \u001b[43m      \u001b[49m\u001b[43mname\u001b[49m\u001b[43m.\u001b[49m\u001b[43mdecode\u001b[49m\u001b[43m(\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mutf-8\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m   1690\u001b[39m \u001b[43m      \u001b[49m\u001b[43mnum_outputs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mnum_outputs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m   1691\u001b[39m \u001b[43m      \u001b[49m\u001b[43minputs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtensor_inputs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m   1692\u001b[39m \u001b[43m      \u001b[49m\u001b[43mattrs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mattrs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m   1693\u001b[39m \u001b[43m      \u001b[49m\u001b[43mctx\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[32m   1694\u001b[39m \u001b[43m  \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m   1695\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m   1696\u001b[39m   outputs = execute.execute_with_cancellation(\n\u001b[32m   1697\u001b[39m       name.decode(\u001b[33m\"\u001b[39m\u001b[33mutf-8\u001b[39m\u001b[33m\"\u001b[39m),\n\u001b[32m   1698\u001b[39m       num_outputs=num_outputs,\n\u001b[32m   (...)\u001b[39m\u001b[32m   1702\u001b[39m       cancellation_manager=cancellation_context,\n\u001b[32m   1703\u001b[39m   )\n",
      "\u001b[36mFile \u001b[39m\u001b[32md:\\Apps\\Python\\Python312\\Lib\\site-packages\\tensorflow\\python\\eager\\execute.py:53\u001b[39m, in \u001b[36mquick_execute\u001b[39m\u001b[34m(op_name, num_outputs, inputs, attrs, ctx, name)\u001b[39m\n\u001b[32m     51\u001b[39m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[32m     52\u001b[39m   ctx.ensure_initialized()\n\u001b[32m---> \u001b[39m\u001b[32m53\u001b[39m   tensors = \u001b[43mpywrap_tfe\u001b[49m\u001b[43m.\u001b[49m\u001b[43mTFE_Py_Execute\u001b[49m\u001b[43m(\u001b[49m\u001b[43mctx\u001b[49m\u001b[43m.\u001b[49m\u001b[43m_handle\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mdevice_name\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mop_name\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m     54\u001b[39m \u001b[43m                                      \u001b[49m\u001b[43minputs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mattrs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mnum_outputs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m     55\u001b[39m \u001b[38;5;28;01mexcept\u001b[39;00m core._NotOkStatusException \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[32m     56\u001b[39m   \u001b[38;5;28;01mif\u001b[39;00m name \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n",
      "\u001b[31mKeyboardInterrupt\u001b[39m: "
     ]
    }
   ],
   "source": [
    "import random\n",
    "import sys\n",
    "\n",
    "for epoch in range(1, 60):\n",
    "    print('epoch', epoch)\n",
    "    # Fit the model for 1 epoch on the available training data\n",
    "    model.fit(x, y,\n",
    "              batch_size=128,\n",
    "              epochs=1)\n",
    "\n",
    "    # Select a text seed at random\n",
    "    start_index = random.randint(0, len(text) - maxlen - 1)\n",
    "    generated_text = text[start_index: start_index + maxlen]\n",
    "    print('--- Generating with seed: \"' + generated_text + '\"')\n",
    "\n",
    "    for temperature in [0.2, 0.5, 1.0, 1.2]:\n",
    "        print('------ temperature:', temperature)\n",
    "        sys.stdout.write(generated_text)\n",
    "\n",
    "        # We generate 400 characters\n",
    "        for i in range(400):\n",
    "            sampled = np.zeros((1, maxlen, len(chars)))\n",
    "            for t, char in enumerate(generated_text):\n",
    "                sampled[0, t, char_indices[char]] = 1.\n",
    "\n",
    "            preds = model.predict(sampled, verbose=0)[0]\n",
    "            next_index = sample(preds, temperature)\n",
    "            next_char = chars[next_index]\n",
    "\n",
    "            generated_text += next_char\n",
    "            generated_text = generated_text[1:]\n",
    "\n",
    "            sys.stdout.write(next_char)\n",
    "            sys.stdout.flush()\n",
    "        print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "As you can see, a low temperature results in extremely repetitive and predictable text, but where local structure is highly realistic: in \n",
    "particular, all words (a word being a local pattern of characters) are real English words. With higher temperatures, the generated text \n",
    "becomes more interesting, surprising, even creative; it may sometimes invent completely new words that sound somewhat plausible (such as \n",
    "\"eterned\" or \"troveration\"). With a high temperature, the local structure starts breaking down and most words look like semi-random strings \n",
    "of characters. Without a doubt, here 0.5 is the most interesting temperature for text generation in this specific setup. Always experiment \n",
    "with multiple sampling strategies! A clever balance between learned structure and randomness is what makes generation interesting.\n",
    "\n",
    "Note that by training a bigger model, longer, on more data, you can achieve generated samples that will look much more coherent and \n",
    "realistic than ours. But of course, don't expect to ever generate any meaningful text, other than by random chance: all we are doing is \n",
    "sampling data from a statistical model of which characters come after which characters. Language is a communication channel, and there is \n",
    "a distinction between what communications are about, and the statistical structure of the messages in which communications are encoded. To \n",
    "evidence this distinction, here is a thought experiment: what if human language did a better job at compressing communications, much like \n",
    "our computers do with most of our digital communications? Then language would be no less meaningful, yet it would lack any intrinsic \n",
    "statistical structure, thus making it impossible to learn a language model like we just did.\n",
    "\n",
    "\n",
    "## Take aways\n",
    "\n",
    "* We can generate discrete sequence data by training a model to predict the next tokens(s) given previous tokens.\n",
    "* In the case of text, such a model is called a \"language model\" and could be based on either words or characters.\n",
    "* Sampling the next token requires balance between adhering to what the model judges likely, and introducing randomness.\n",
    "* One way to handle this is the notion of _softmax temperature_. Always experiment with different temperatures to find the \"right\" one."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
